1. Field of the Invention
The present invention relates to apparatuses, methods and computer program products for separating and merging a coded moving picture sequence signal, and more particularly, to apparatuses, methods and computer program products for transcoding a first coded moving picture sequence signal to separate into and generate a second coded moving picture sequence signal and a differential coded moving picture sequence signal, which is a difference between the first coded moving picture sequence signal and the second coded moving picture sequence signal, and merging the second coded moving picture sequence signal and the differential coded moving picture sequence signal to reconstruct the first coded moving picture sequence signal.
2. Description of the Related Art
There have so far been proposed a wide variety of systems for compressing and encoding a moving picture having a considerable amount of data to produce a coded moving picture sequence signal. The international standard, ISO-IEC 13818, was created for a system operable to encode a digital video signal with an associated digital audio signal and commonly called “Moving Picture Expert Group Phase 2”, i.e., “MPEG-2”. In such an encoding system, the coded moving picture sequence signal is outputted in the form of bit streams. In particular, the bit streams conformable to the above MPEG-2 standard will be referred to as “MPEG-2 bit streams” hereinlater. Recently, the system of this type becomes more utilizable for various technical fields, such as a communications system, a television broadcasting service system, and so on.
The above MPEG-2 bit stream have a hierarchical structure consisting of: in turn, a top, sequence layer; a GROUP OF PICTURES layer; a picture layer; a slice layer; a macroblock layer; and a low, block layer.
The typical encoder operates under the MPEG-2 standard through a method of compressing and encoding a moving picture as follows. The method comprises the steps of:    (a) inputting the moving picture sequence consisting of a series of pictures;    (b) temporarily storing the series of pictures as frames in memories, respectively;    (c) computing a difference between one frame and another frame to eliminate redundancy in a time axis direction; and    (d) orthogonal transforming, e.g., discrete cosine transforming (DCT), a plurality of picture elements within each of the frames to eliminate redundancy in a spatial axis direction.
The encoder thus constructed can compress and encode the moving picture to generate and output a coded moving picture sequence signal in the form of the MPEG-2 bit stream through a transmitting path at a predetermined bit rate. The coded moving picture sequence signal is then transmitted from the encoder to a decoder which is operated to decode the coded signal to reproduce the moving picture. The typical decoder is operated to decode the coded moving picture sequence signal through a so-called bi-directionally predicting method which comprises the steps of:    (a) storing one reproduced picture, generally referred to as “intra-picture”, i.e., “I-picture”, in a first frame memory;    (b) estimating another picture generally referred to as “predictive-picture”, i.e., “P-picture”, followed by the I-picture, on the basis of the information on the difference between the I-picture and P-picture;    (c) storing the estimated P-picture in a second frame memory; and    (d) estimating further another picture interposed between the I-picture and P-picture, generally referred to as “bi-directionally predictive-picture”, i.e., “B-picture”.
Here, the I-picture is encoded independently of the pictures of the other types, so that an I-picture can be reproduced as a single static image only by itself. A P-picture can be predicted on the basis of the I-picture or another P-picture located on a position prior to the P-picture to be encoded. I-picture is referred to as “intra-picture” while P-picture and B-picture are referred to as “inter-pictures”.
In the above encoder, the amount of information on the coded moving picture sequence signal is, however, variable. In particular, the amount of information increases remarkably when a scene is changed. The decoder is generally provided with an input buffer for receiving the coded moving picture sequence signal from the encoder. The input buffer of the decoder, however, has a limited storage capacity. Therefore, when a large number of bits of the coded moving picture sequence signal are transmitted from the encoder to the decoder, the input buffer overflows with the bits of the coded moving picture sequence signal thereby making the decoder difficult to process the coded moving picture sequence signal. In order to transmit such coded moving picture sequence signal having a variable number of bits through the transmitting path at a predetermined bit rate and to make it possible for any decoder to receive the whole of the coded moving picture sequence signal without overflow, the encoder comprises: an output buffer for temporarily storing the coded moving picture sequence signal before transmitting the coded moving picture sequence signal through the transmitting path; and a rate controller for controlling the amount of bits of the coded moving picture sequence signal stored in the output buffer so as to keep the amount of bits of the coded moving picture sequence signal to be transmitted to the decoder for a predetermined time from exceeding the capacity of the input buffer of the decoder, thereby controlling the bit rate of the coded moving picture sequence signal.
A typical rate controlling method in MPEG-2 standard is described in “ISO-IEC/JTC1/SC29/WG11/N0400 Test Model 5”, April, 1993, hereinlater referred to as “TM-5”. The rate controlling method according to the TM-5 comprises the steps of:    (I) allocating a target number of bits to a picture of each type on the basis of the total number of bits, i.e., R, available to the pictures to be encoded in the GROUP OF PICTURES;    (II) computing the reference value of a quantization parameter used for the quantization of each of macroblocks in the picture on the basis of the utilization capacity of a “virtual buffer” to perform the rate control; and    (III) modulating the reference value of the quantization parameter in accordance with the spatial activity in the macroblock.
Furthermore, there are many types of decoders. For instance, a decoder is designed to decode the coded signal in a unique compression format different from that of the MPEG-2 bit stream, and another decoder is connectable to a transmitting path having a different bit rate. The decoder of those types is therefore required to provide with an apparatus, a so-called transcoder, for converting the MPEG-2 bit streams into another appropriate coded signal in a specified format having a required bit rate. The transcoder makes it possible for the encoder to transmit the coded signal to any types of decoders.
Referring to FIG. 14 of the drawings, there is shown a transcoder of one typical type as a first conventional transcoder 50. The conventional transcoder 50 has an input terminal a1 electrically connected to a first transmitting path, not shown, and an output terminal a2 electrically connected to a second transmitting path, not shown. The conventional transcoder 50 is designed to input first bit streams b1 at a predetermined input bit rate through the input terminal a1, to convert the first bit streams b1 into second bit streams b2 to be outputted at a predetermined output bit rate, i.e., a target bit rate, lower than the input bit rate of the inputted first bit streams b1, and then to output the second bit streams b2 through the output terminal a2. The conventional transcoder 50 comprises a variable length decoder 51, referred to as “VLD” in the drawings, an inverse quantizer 53, referred to as “IQ” in the drawings, a quantizer 55, referred to as “Q” in the drawings, a variable length encoder 57, referred to as “VLC” in the drawings, and a rate controller 59.
The variable length decoder 51 is electrically connected to the input terminal a1 and designed to decode a coded moving picture sequence signal within the first bit streams b1 inputted through the input terminal a1 to reconstruct original picture data for each of pictures including a matrix of original quantization coefficients, referred to as “level”, for each of macroblocks within each of the pictures and an original quantization parameter, hereinlater referred to as “first quantization parameter Q1”.
The inverse quantizer 53 is electrically connected to the variable length decoder 51 and designed to input the matrix of original quantization coefficients level from the variable length decoder 51 and the first quantization parameter Q1. The inverse quantizer 53 is further designed to inversely quantize the inputted matrix of original quantization coefficients level with the first quantization parameter Q1 to generate a matrix of inverse-quantization coefficients, referred to as “dequant”, i.e., DCT coefficients, for each of macroblocks as follows:                     dequant        =                              {                                          2                ×                level                            +                              sign                ⁡                                  (                  level                  )                                                      }                    ×                                                    Q                1                            ×              QM                        32                                              equation        ⁢                                   ⁢                  (          a1          )                                or                                                       dequant        =                  level          ×                                                    Q                1                            ×              QM                        16                                              equation        ⁢                                   ⁢                  (          a2          )                    
where the equation (a1) is used for the intra-picture while the equation (a2) is used for the inter-picture. QM is a matrix of quantization parameters stored in a predetermined quantization table. The first quantization parameter Q1 and the matrix of quantization parameters QM are derived from the inputted first bit streams b1 by the decoder 51. Here, the original quantization coefficients level, the inverse-quantization coefficients dequant, the matrix of quantization parameters QM, and the first quantization parameter Q1 are integers. The inverse-quantization coefficients dequant calculated by the equations (a1) and (a2) should be rounded down to the nearest integer.
The quantizer 55 is electrically connected to the inverse quantizer 53 and designed to input the matrix of inverse-quantization coefficients dequant from the inverse quantizer 53 and then quantize the inputted matrix of inverse-quantization coefficients dequant for each of macroblocks with a second quantization parameter, referred to as “Q2” hereinlater, to generate a matrix of re-quantization coefficients, referred to as “tlevel”, as follows:                     tlevel        =                  dequant          ×                      16                                          Q                2                            ×              QM                                                          equation        ⁢                                   ⁢                  (          a3          )                                or                                                       tlevel        =                              dequant            ×                          16                                                Q                  2                                ×                QM                                              +                                    sign              ⁡                              (                dequant                )                                      ×                          1              2                                                          equation        ⁢                                   ⁢                  (          a4          )                    
where the equation (a3) is used for the inter-picture, while the equation (a4) is used for the intra-picture. The second quantization parameter Q2 is obtained by the rate controller 59. Here, the re-quantization coefficients tlevel and the second quantization parameter Q2 are also integers. The re-quantization coefficients tlevel calculated by the equations (a3) and (a4) should be rounded down to the nearest integer. Such rounding operation for the integers will be omitted from the later description for avoiding tedious repetition.
The variable length encoder 57 is electrically connected to the quantizer 55 and designed to input the re-quantization coefficients tlevel from the quantizer 55 and then encode the inputted matrix of the re-quantization coefficients tlevel to generate objective picture data for each of pictures to sequentially output the objective picture data in the form of the second bit streams b2 through the output terminal a2. The variable length encoder 57 is further electrically connected to the variable length decoder 51 and designed to input a diversity of information data included in the first bit streams b1 necessary for the second bit streams b2 from the variable length decoder 51.
The rate controller 59 is electrically connected to the inverse quantizer 53 and designed to perform rate control process in accordance with the TM-5 on the basis of the information obtained from the inverse quantizer 53 as described below.
Referring to FIG. 15 of the drawings, there is shown a flowchart of the rate controlling process in accordance with the TM-5 carried out in the conventional transcoder 50. As shown in FIG. 15, the rate controlling process comprises steps A1 to A14.
In the step A1, “1” is assigned to a picture number variable n representing the serial number of a picture within the first bit streams b1. Hereinlater, a n-th picture in the first bit streams b1 is referred to as “pic(n)”.
In the following step A2, a global complexity measure, referred to as Xi, Xp, or Xb, for a picture of the corresponding type, i.e., I, P or B-picture is computed as follows:Xi=Si×Qi  equation (a5)orXp=Sp×Qp  equation (a6)orXb=Sb×Qb  equation (a7)where Si, Sp, or Sb is the number of bits generated for an encoded I, P or B-picture, and Qi, Qp, or Qb is the average quantization parameter computed by averaging the actual quantization values used during the quantization of the all macroblocks within I, P or B-picture. The average quantization parameters Qi, Qp, and Qb are normalized within a range of 1 to 31. The average quantization parameters Qi, Qp, and Qb respectively correspond to the first quantization parameters Q1 obtained from the variable length decoder 51.
The global complexity measure Xi, Xp, or Xb of the corresponding picture is inversely proportional to the compressing ratio of the moving picture, namely, the ratio of the amount of information in the second bit streams b2 to that in the first bit streams b1. Namely, as the amount of information in the first bit streams b1 becomes larger, the compressing ratio is decreased. Therefore, the global complexity measure Xi, Xp, or Xb of the corresponding picture becomes larger, as the compressing ratio is decreased. In contrast, the global complexity measure Xi, Xp, or Xb of the corresponding picture becomes smaller, as the compressing ratio is increased.
The initial value of global complexity measure Xi, Xp, or Xb of the corresponding picture is given as follows:Xi=160×Target_Bitrate/115  equation (a8)orXp=60×Target_Bitrate/115  equation (a9)orXb=42×Target_Bitrate/115  equation (a10)
where Target_Bitrate is measured in bits/s and corresponds to the target bit rate of the first conventional transcoder 50.
In the following step A3, the target number of bits for a picture of the corresponding type, i.e., I, P or B-picture to be encoded in the current GROUP OF PICTURES, referred to as Ti, Tp, or Tb is computed as:                               T          i                =                              R                          1              +                                                                    N                    p                                    ⁢                                      X                    p                                                                                        X                    i                                    ⁢                                      K                    p                                                              +                                                                    N                    b                                    ⁢                                      X                    b                                                                                        X                    i                                    ⁢                                      K                    b                                                                                ⁢                                           ⁢          or                                    equation        ⁢                                   ⁢                  (          a11          )                                                  T          p                =                              R                                          N                p                            +                                                                    N                    b                                    ⁢                                      K                    p                                    ⁢                                      X                    b                                                                                        K                    b                                    ⁢                                      X                    p                                                                                ⁢                                           ⁢          or                                    equation        ⁢                                   ⁢                  (          a12          )                                                  T          b                =                  R                                    N              b                        +                                                            N                  p                                ⁢                                  K                  b                                ⁢                                  X                  p                                                                              K                  p                                ⁢                                  X                  b                                                                                        equation        ⁢                                   ⁢                  (          a13          )                    
where Np and Nb are the number of P-pictures and B-pictures remained not yet encoded in the current GROUP OF PICTURES, respectively. Kp and Kb are constants computed on the basis of the ratio of the quantization value of P-picture to the quantization value of I-picture, and the ratio of the quantization parameter of B-picture to the quantization value of I-picture, respectively. It is assumed that the quality of the whole image will be always optimized with Kp=1.0 and Kb=1.4.
In the following step A4, it is judged upon whether the picture number variable n is “1” or not, i.e., the current picture is the first picture pic(1) or not. When it is judged that the picture number variable n is “1”, i.e., the current picture is the first picture pic(1), the step A4 goes forward to the step A5. When, on the other hand, it is judged that the picture number variable n is not “1”, i.e., the current picture is not the first picture, the step A4 goes forward to the step A6. In the step A5, the total number of bits available to the pictures to be encoded in the current GROUP OF PICTURES, i.e., the remaining number of bits available to the GROUP OF PICTURES, hereinlater referred to as R, is initialized in accordance with the following equation (a14). This remaining number of bits available to the GROUP OF PICTURES R before encoding the first picture pic(1) within the GROUP OF PICTURES is computed as follows:R=Target_Bitrate×NPIC/picture_rate+R  equation (a14)
where NPIC is the total number of pictures of any type in the GROUP OF PICTURES, and picture_rate is expressed in the number of pictures decoded and indicated per second. At the start of the sequence R=0.
In the step A6, the above remaining number of bits available to the GROUP OF PICTURES R before encoding the current picture pic(n) is updated as follows:R=R−Si  equation (a15) orR=R−Sp  equation (a16)orR=R−Sb  equation (a17)
where Si, Sp, or Sb is the number of bits generated in the previously encoded picture pic(n−1) of the corresponding type (I, P or B).
The step A5 or A6 goes forward to the step A7 wherein “1” is assigned to a macroblock number variable j (j>=1) representing the serial number of a macroblock within one of the pictures. Hereinlater, the j-th macroblock in the picture is referred to as “MB(j)”.
In the following step A8, a utilization volume of the capacity of a virtual buffer for I, P or B-pictures, referred to as di(j), dp(j) or db(j), before encoding the macroblock MB(j) is computed as follows:                                           d            i                    ⁡                      (            j            )                          =                                            d              i                        ⁡                          (              0              )                                +                      B            ⁡                          (                              j                -                1                            )                                -                                                                      T                  i                                ×                                  (                                      j                    -                    1                                    )                                            NMB                        ⁢                                                   ⁢            or                                              equation        ⁢                                   ⁢                  (          a18          )                                                              d            p                    ⁡                      (            j            )                          =                                            d              p                        ⁡                          (              0              )                                +                      B            ⁡                          (                              j                -                1                            )                                -                                                                      T                  p                                ×                                  (                                      j                    -                    1                                    )                                            NMB                        ⁢                                                   ⁢            or                                              equation        ⁢                                   ⁢                  (          a19          )                                                              d            b                    ⁡                      (            j            )                          =                                            d              b                        ⁡                          (              0              )                                +                      B            ⁡                          (                              j                -                1                            )                                -                                                    T                b                            ×                              (                                  j                  -                  1                                )                                      NMB                                              equation        ⁢                                   ⁢                  (          a20          )                    
where B(j−1) is the total number of bits generated for encoded macroblocks in the picture up to and including the (j−1)th macroblock MB(j−1). NMB is the total number of macroblocks in the picture. di(j), dp(j), or db(j) is the utilization volume of the capacity of the virtual buffer at the j-th macroblock MB(j) for I, P, or B-picture.
di(0), dp(0), or db(0) is the initial utilization volume of the virtual buffer for I, P, or B-picture and given by:di(0)=10×r/31  equation (a21)ordp(0)=Kp×di(0)  equation (a22)ordb(0)=Kb×di(0)  equation (a23)
where r is referred to as “reaction parameter” and used for the control of the reaction rate of the feed back loop as follows:r=2×Target_Bitrate/picture_rate  equation (a24)
The final utilization volume of the virtual buffer, referred to as, di(NMB), dp(NMB), or db(NMB) of the last macroblock, i.e., NMB-th macroblock MB(NMB) of the current picture pic(n) will be used as the initial utilization volume of the virtual buffer for I, P, or B-picture, i.e., di(0), dp(0), or db(0) of the same type to encode the first macroblock MB(j) within the next picture pic(n+1).
In the following step A9, the reference quantization parameter Q(j) of the j-th macroblock MB(j) for each of the pictures is computed on the basis of the aforesaid utilization volume of the virtual buffer, i.e., d(j) as follows:Q(j)=d(j)×31/r  equation (a25)
Here, the reference quantization parameter Q(j) is identical with the aforesaid second quantization parameter Q2 of the j-th macroblock MB(j).
In the following step A10, the j-th macroblock MB(j) is quantized with the reference quantization parameter Q(j) computed in the step A9. In the following step A11, the macroblock number variable j is incremented by one. The step A11 goes forward to the step A12 wherein it is judged upon whether the macroblock number variable j is more than the total number of macroblocks NMB within the n-th picture pic(n) or not. When it is judged that the macroblock number variable j is not more than the total number of macroblocks NMB within the n-th picture pic(n), the step A12 returns to the step A8. When, on the other hand, it is judged that the macroblock number variable j is more than the total number of macroblocks NMB within the n-th picture pic(n), the step A12 goes forward to the step A13.
The macroblock number variable j thus serves as a loop counter for repeating the process from the steps A8 to A11 to encode all the macroblocks from the 1st macroblock MB(1) up to the j-th macroblock MB(j) in the present picture pic(n). The entire macroblocks starting from the first macroblock MB(1) up to the NMB-th macroblock MB(NMB) in the n-th picture pic(n) can be thus encoded sequentially.
In the step A13, the picture number variable n is incremented by one. Then the step A13 goes forward to the step A14 wherein it is judged upon whether the picture number variable n is more than the total number of pictures, i.e., NPIC or not. When it is judged that the picture number variable n is not more than the total number of pictures, NPIC, the step A14 returns to the step A2. When, on the other hand, it is judged that the picture number variable n is more than the total number of pictures, NPIC, this routine of the rate controlling process is terminated. The picture number variable n thus serves as a loop counter for repeating the process from steps A2 to A13 to process all the pictures from the first picture pic(1) to the n-th picture pic(n) in the present GROUP OF PICTURES. The entire pictures starting from the first picture pic(1) up to the NPIC-th picture pic(NPIC), in the present GROUP OF PICTURES can be therefore processed sequentially.
The aforesaid conventional transcoder 50, however, can obtain no information on the structure of GROUP OF PICTURES such as a picture cycle of I or P-pictures within each of the GROUP OF PICTURES, so that the transcoder 50 must estimate the structure of GROUP OF PICTURES within the inputted moving picture sequence signal to allocate the number of bits to pictures of each type within the estimated structure of GROUP OF PICTURES.
Furthermore, the first conventional transcoder 50 is required to decode the first bit streams b1 almost all over the layers such as the sequence layer, the GROUP OF PICTURES layer, the picture layer, the slice layer and the macroblock layer in order to derive necessary data for transcoding the first bit streams b1 into the second bit streams b2. The operation takes time, thereby causing the delay in the transcoding process.
Referring to FIG. 16 of the drawings, there is shown an improvement of the above transcoder 50 as a second conventional transcoder 60. The second conventional transcoder 60 is operated to perform the rate control without estimating the structure of GROUP OF PICTURES. As shown in FIG. 16, the second conventional transcoder 60 comprises a delay circuit 61 and a rate controller 62 in addition to the variable length decoder 51, the inverse quantizer 53, the quantizer 55 and the variable length encoder 57 same as those of the first conventional transcoder 50 shown in FIG. 14. The same constitutional elements are simply represented by the same reference numerals as those of the conventional transcoder 50, and will be thus omitted from description for avoiding tedious repetition.
The delay circuit 61 is interposed between the variable length decoder 51 and the inverse quantizer 53 and designed to control the flow of the signal from the variable length decoder 51 to the inverse quantizer 53. The delay circuit 61 is operated to delay the operation start time of the inverse quantizer 53 so that the inverse quantizer 53 does not start the inverse-quantizing process until the variable length decoder 51 terminates the process of decoding one of the pictures in the coded moving picture sequence signal.
As shown in FIG. 16, the rate controller 62 of the second conventional transcoder 60 includes a target ratio computing unit 63, an input bit summing unit 65, a bit difference computing unit 67, a target output bit updating unit 69, and a quantization parameter computing unit 71.
The target ratio computing unit 63 is electrically connected to the variable length decoder 51 and designed to input an input bit rate of the first bit streams b1, hereinlater referred to as “Input_Bitrate”, from the variable length decoder 51, and input a target bit rate, hereinlater referred to as “Target_Bitrate” through a terminal a3. Alternatively, the target bit rate Target_Bitrate may have been stored in an internal memory, or determined on the basis of internal switches. The target ratio computing unit 63 is designed to then compute a target ratio, hereinlater referred to as “ioRatio” of the target bit rate Target_Bitrate to the input bit rate Input_Bitrate for each of pictures as follows:                     ioRatio        =                  Target_Bitrate          Input_Bitrate                                    equation        ⁢                                   ⁢                  (          a26          )                    
The input bit summing unit 65 is designed to sum up the number of inputting bits of the picture decoded by the variable length decoder 51 to produce the total number of inputting bits, hereinlater referred to as “Tin”. On the other hand, the target output bit updating unit 69 is designed to compute a target number of outputting bits to be generated by the variable length encoder 57, hereinlater referred to as “Tout”. The target number of outputting bits Tout is computed by multiplying the total number of inputting bits Tin by the target ratio ioRatio as follows:Tout=Tin×ioRatio  equation (a27)
The bit difference computing unit 67 is electrically connected to the variable length encoder 57 and the target output bit updating unit 69, and designed to input a real number of outputting bits encoded by the variable length encoder 57, hereinlater referred to as “Treal”, and input the target number of outputting bits Tout. The bit difference computing unit 67 is designed to then compute a difference between the target number of outputting bits Tout and the real number of outputting bits Treal, hereinlater referred to as a “difference number of bits”, i.e., “Tdiff” as follows:Tdiff=Treal−Tout  equation (a28)
The target output bit updating unit 69 is electrically connected to the target ratio computing unit 63, the input bit summing unit 65, and the bit difference computing unit 67. The target output bit updating unit 69 is designed to update the target number of outputting bits Tout on the basis of the difference number of bits Tdiff as follows:Tout=Tout−Tdiff  equation (a29)
The quantization parameter computing unit 71 is electrically connected to the target output bit updating unit 69 and designed to compute the reference quantization parameter Q(j) for each of macroblocks MB(j) on the basis of the target outputting bits Tout updated by the target output bit updating unit 69 in accordance with the step II of the TM-5.
FIG. 17 shows the flowchart of the rate controlling process performed by the above conventional transcoder 60. The rate controlling process performed in the transcoder 60 comprises the steps B1 to B13. The steps B6 to B13 are almost the same as those of the steps A7 to A14, respectively, in the rate controlling process shown in FIG. 15 except for the step B7 wherein the utilization volume of the capacity of the virtual buffer is computed on the basis of the target number of outputting bits Tout given by the target output bit updating unit 69 instead of the target number of bits Ti, Tp or Tb computed in the step A3 shown in FIG. 15. The same steps will be thus omitted from description for avoiding tedious repetition.
In the step B1, “1” is assigned to the picture number variable n. The step B1 then goes forward to the step B2 wherein the target ratio ioRatio is computed by the above equation (a26). In the following step B3, the difference number of bits Tdiff is computed for the present picture pic(n) by the above equation (a28). The step B3 then goes forward to the step B4 wherein the number of inputting bits Tin is summed up within the first bit streams b1. In the step B5, the target number of outputting bits Tout is computed by the above equation (a27), and further updated by the above equation (a29).
In the second conventional transcoder 60 thus constructed, the inverse quantizer 53, however, cannot start the inverse-quantization process until the target transcoding frame is completely decoded, thereby causing the delay in the transcoding process.
Referring to FIGS. 18 and 19 of the drawings, there is shown another improvement of the above transcoder 50 as a third conventional transcoder 80. The third conventional transcoder 80 is also adaptable to perform the rate control without estimating the structure of GROUP OF PICTURES. As shown in FIG. 18, the third conventional transcoder 80 comprises an input terminal a1 electrically connected to a first transmitting path and designed to input an input bit streams b3 at the input bit rate, and an output terminal a2 electrically connected to a second transmitting path and designed to output an output bit streams b4 at the target bit rate. In the third conventional transcoder 80, the input bit streams b3 may have a format, non-adaptable for the MPEG-2, different from that of the bit streams b1 of the first and second conventional transcoders 50 and 60. The input bit streams b3 have information on the number of coding bits previously recorded thereon by the encoder, not shown.
The third conventional transcoder 80 comprises a variable length decoder 81 electrically connected to the input terminal a1, and a rate controller 82 in addition to the inverse quantizer 53, the quantizer 55, and the variable length encoder 57 which are same in construction as those of the second transcoder 60 shown in FIG. 16. The rate controller 82 includes a target output bit updating unit 83, and a quantization parameter computing unit 85 in addition to the target ratio computing unit 63, and the bit difference computing unit 67 which are same as those of the second transcoder 60 shown in FIG. 16.
The third conventional transcoder 80 thus constructed can perform the rate control on the basis of the formation on the number of coding bits previously recorded in the input bit streams b3. The variable length decoder 81 is operated to decode the coded moving picture sequence signal within the third bit streams b3 to reconstruct the pictures and the information on the number of coding bits, and transmit the information to the inverse quantizer 53. The variable length decoder 81 is also operated to transmit the number of inputting bits Tin to the target output bit updating unit 83.
The outputting bit updating unit 83 is designed to compute the target number of outputting bits Tout on the basis of the number of inputting bits Tin and the target ratio ioRatio by the above equation (a26). The quantization parameter computing unit 85 is designed to compute the reference quantization parameter Q(j) of the macroblocks MB(j) for each of pictures on the basis of the target number of outputting bits Tout updated by the outputting bit updating unit 83 in accordance with the step II in the TM-5. The quantizer 55 is then operated to quantize the j-th macroblock MB(j) on the basis of the reference quantization parameter Q(j) given by the quantization parameter computing unit 85.
FIG. 19 shows the flowchart of the rate controlling process performed by the above third conventional transcoder 80. The rate controlling process performed in the transcoder 80 comprises the steps C1 to C13. All the steps C1 to C13 are the same as those of the steps B1 to B13, respectively, in the rate controlling process shown in FIG. 17 except for the step C4 wherein the number of inputting bits Tin in the current picture pic(n) is derived from the third bit streams b3 by the decoder 81 to compute the total number of inputting bits Tin.
The third conventional transcoder 80 thus constructed has information on the number of coding bits previously recorded in the third bit streams b3 thereby making it possible to solve the problem of the delay in the second conventional transcoder 60. The third conventional transcoder 80, however, encounters another problem to restrict the form of the inputted bit streams. Moreover, the encoder which is linked with the third transcoder 80 must provide with the above information on the number of coding bits to be recorded in the bit streams, thereby causing the delay of process in the encoder.
In the conventional transcoders 50, 60 and 80, the matrix of the inverse-quantization coefficients dequant is necessary for only the quantizer 55, but unnecessary for the transcoder itself to generate the desired bit streams. In order to eliminate the redundant matrix of the inverse-quantization coefficients dequant, there is proposed a fourth conventional transcoder 90 comprising a level converter 91 instead of the inverse quantizer 53 and the quantizer 55 of the transcoder 50, as shown in FIG. 20.
The level converter 91 is interposed between the variable length decoder 51 and the variable length encoder 57. The level converter 91 is designed to input the original picture data for each of pictures. The original picture data includes a matrix of original quantization coefficients level for each of macroblocks within the corresponding picture. The level converter 91 is electrically connected to the rate controller 59 and designed to input the second quantization parameter Q2 from the rate controller 59.
The level converter 91 is further designed to convert the original picture data for each of pictures including the matrix of original quantization coefficients level into the objective picture data including the matrix of re-quantization coefficients tlevel without generating the matrix of the inverse-quantization coefficients dequant. The following equations (30a) and (31a) for the matrix of re-quantization coefficients tlevel are lead by eliminating the matrix of the inverse-quantization coefficients dequant from the above equations (a1), (a2), (a3) and (a4).                     tlevel        =                  {                                    (                              level                +                                                      sign                    ⁡                                          (                      level                      )                                                        ×                                      1                    2                                                              }                        ×                                          Q                1                                            Q                2                                                                        equation        ⁢                                   ⁢                  (                      30a                    )                                or                                                       tlevel        =                              level            ×                                          Q                1                                            Q                2                                              +                                    sign              ⁡                              (                level                )                                      2                                              equation        ⁢                                   ⁢                  (                      31a                    )                    
where the above equation (30a) is used for the inter-picture, while the above equation (31a) is used for the intra-picture. The level converter 91 is thus operable to convert the original picture data, for each of pictures, into the second picture data with the first quantization parameter Q1 and the second quantization parameter Q2. The first quantization parameter Q1 is decoded from the first bit streams b1 by the variable length decoder 51, while the second quantization parameter Q2 is obtained from the rate controller 59.
In the fourth conventional transcoder 90, the rate controller 59 is designed to perform the rate control over the encoding process in the transcoder 90 according to the TM-5. The variable length encoder 57 is electrically connected to the level converter 91 and to input the above matrix of re-quantization coefficients tlevel from the level converter 91.
The fourth conventional transcoder 90 thus constructed can efficiently perform the transcoding process at high speed without storing the matrix of inverse-quantization coefficients dequant in a memory.
The above conventional transcoders 50, 60, 80 and 90, however, encounter another problem with the rate-distortion performance in converting the quantization level occurred as a result of the re-quantization operation since they cannot obtain all the information on original picture data before the quantization operation. In short, the rate-distortion performance in converting the quantization level is unstable and variable in accordance with the first and second quantization parameters and the level of the original quantization coefficients level. Therefore, as the amount of reduced information becomes larger, the quantization error is liable to increase, thereby causing unstable rate control in transcoding.
The applicant of the present application filed a U.S. patent application Ser. No. 09/604,973 on Jun. 28, 2000.
The applicant disclosed therein an apparatus, a method and a computer program product for transcoding a coded moving picture sequence signal, being operable to compute an optimized re-quantization parameter on the basis of the inverse-quantization parameter and the previously computed re-quantization parameter in consideration of the characteristics of the rate-distortion performance dependent on the re-quantization parameter and the inverse-quantization parameter.
The apparatus disclosed therein comprising an inverse quantizer for performing the inverse-quantization operation and a quantizer for performing the quantization operation, is characterized in that the apparatus further comprises quantization parameter switching means for switching the quantization parameter in consideration of the characteristics of the rate-distortion performance dependent on the inputted quantization parameter, thereby making it possible for the apparatus to minimize the quantization error occurred when the matrix of original quantization coefficients is transformed to the matrix of re-quantization coefficients.
In the meantime, different techniques have been found such as data partitioning and SNR scalability for dividing picture signals conveying picture information into two separate picture signals consisting of base layer picture signal indicative of basic picture information and enhancement layer picture signal indicative of high quality picture information in order to prevent the quality of picture from deteriorating.
More particularly, the data partitioning provides a method of dividing bit streams conveying picture information into two separate bit streams consisting of base layer bit streams conveying low-frequency DCT coefficients and enhancement layer bit streams conveying high-frequency DCT coefficients before encoding, and the base layer bit streams and enhancement layer bit streams thus divided are recombined before decoding. Original picture information can be roughly decoded and reproduced from the base layer bit streams conveying low-frequency DCT coefficients, but not from the enhancement layer bit streams conveying high-frequency DCT coefficients alone. The high-quality original picture information can be decoded and reproduced from the recombination of the base layer bit streams conveying low-frequency DCT coefficients and the enhancement layer bit streams conveying high-frequency DCT coefficients.
The SNR scalability provides a method of dividing picture signals containing picture information into two separate picture signals consisting of base layer picture signals containing low-SNR image information and enhancement layer picture signals containing auxiliary information before encoding. The method of SNR scalability will be described in detail. The original picture signals have original DCT coefficients. The quantizer is operated to roughly quantize base layer bit picture signals containing low-SNR image information to generate low-SNR bit streams. The inverse quantizer is operated to inversely quantize the low-SNR bit streams thus generated to roughly reproduce DCT coefficients. Then, the difference information between the original DCT coefficients and the reproduced DCT coefficients is extracted and quantized to generate the enhancement layer picture signals. The enhancement layer picture signals thus generated are used as auxiliary information in combination with the base layer picture signals (low-SNR signals) to reproduce high-SNR signals.
The above described methods, however, encounter a problem of reducing the quality of service, i.e., QoS. The transcoding process above described is non-reversible. The transcoder, in general, is operated to decode and inversely quantize DCT coefficients of input bit streams and re-quantize the DCT coefficients thus inversely quantized with re-quantization parameters greater then the original quantization parameters to reduce the bit rate. This means that the input bit streams before the transcoding operation cannot be reproduced from the transcoded bit streams. This leads to the fact that the QoS for the input bit streams cannot be reproduced.
The data partitioning method permits to divide bit streams into two separate bit streams consisting of base layer bit streams indicative of low-frequency DCT coefficients and enhancement layer bit streams indicative of high-frequency DCT coefficients before encoding. There is, however, provided no method of dividing MPEG-2 bit streams in conformance with MP@ML, which are not in a hierarchical structure, into base layer bit streams and enhancement layer bit streams. Furthermore, although the data partitioning method may permit to divide bit streams into the base layer bit streams and enhancement layer bit streams before encoding, a decoder conforming to MP@ML cannot decode the base layer bit streams and enhancement layer bit streams thus divided.
According to the syntax of the data partitioning, the code specifying a boundary between low-frequency coefficients and high-frequency coefficients is defined as “Priority_break_point”, which makes it possible for a decoder to distinguish the low-frequency coefficients from the high-frequency coefficients. The decoder conforming to MP@ML, on the other hand, cannot recognize “Priority_break_point”. Furthermore, the bit streams indicative of low-frequency coefficients include no EOB code, thereby making it impossible for the MP@ML decoder to reproduce the bit streams indicative of low-frequency coefficients. This leads to the fact that the data partitioning method requires a decoder dedicated to the data partitioning method in place of the decoder conforming to MP@ML.
Similarly to the data partitioning, the SNR scalability method permits to divide bit streams into two separate bit streams consisting of base layer bit streams containing low-SNR signals and enhancement layer bit streams containing the auxiliary information before encoding. An encoder conforming to MP@ML, however, cannot divide bit streams into base layer bit streams containing low-SNR signals and enhancement layer bit streams containing the auxiliary information and encode the base layer bit streams and enhancement layer bit streams thus divided. Nor can a decoder conforming to MP@ML decode the base layer bit streams and the enhancement layer bit streams. This leads to the fact that the SNR scalability method requires an encoder and a decoder dedicated to the SNR scalability in place of the encoder and decoder conforming to MP@ML.
Furthermore, the base layer bit streams and the enhancement layer bit streams are required to be processed in parallel, thereby making it complex and difficult to design such SNR scalability conformable encoder and decoder. Moreover, the SNR scalability conformable decoder is operated to receive the base layer bit streams and the enhancement layer bit streams to reproduce and output original picture signals but not in the form of bit streams. This means that the picture signal thus reproduced and outputted must be transcoded again if it is required be in the form of bit streams.
That the above data partitioning and SNR scalability operations require respective dedicated encoders and decoders is attributed to the fact that the respective decoders and encoders are operative to perform the process of dividing bit streams into base layer bit streams and the enhancement layer bit streams, and the process of recombining the base layer bit streams and the enhancement layer bit streams to reconstruct original bit streams.